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ABSTRACT 


During  the  2005  North  Atlantic  hurricane  season,  an 
objective  tropical  cyclone  vortex  identification  and 
tracking  technique  was  applied  to  analyzed  and  forecast 
fields  of  three  global  operational  numerical  models  -  the 
National  Centers  for  Environmental  Prediction  Global 
Forecast  System  (GFS) ,  the  Navy  Operational  Global 
Atmospheric  Prediction  System  (NOGAPS) ,  and  the  United 
Kingdom  Meteorological  Office  model  (UKMET) .  For  the 
purpose  of  evaluating  each  model's  performance  with  respect 
to  forecasting  tropical  cyclone  formation,  14  relevant 
parameters  are  cataloged  for  every  tropical  vortex. 

In  this  study,  nine  of  the  fourteen  parameters  are 
subjected  to  a  linear  discriminant  analysis  applied  to  all 
forecast  vortices  that  exceed  vorticity  and  warm  core 
thresholds.  The  goal  is  to  determine  the  combination  of 
parameters  for  each  model,  at  each  12-h  forecast  period  to 
120h,  that  best  discriminates  between  a  vortex  that  is 
correctly  forecast  to  intensify  into  a  tropical  cyclone 
(developer)  and  a  vortex  that  is  forecast  to  intensify  into 
a  tropical  cyclone,  but  does  not  (false  alarm) .  The 
performance  of  the  resulting  discriminant  functions  are 
then  assessed  using  the  Heidke  Skill  Score  and  Receiver 
Operating  Characteristic  curves.  Overall,  the  methodology 
applied  to  forecasts  from  the  UKMET  model  shows  the  most 
skill  with  regard  to  identifying  correct  forecasts  of 
tropical  cyclone  formation. 
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I. 


INTRODUCTION 


Improvements  in  operational  global  models  have 
resulted  in  increased  accuracy  of  72-h  tropical  cyclone 
track  forecasts.  In  an  effort  to  extend  preparation  lead- 
time,  the  National  Hurricane  Center  (NHC)  and  the  Joint 

Typhoon  Warning  Center  (JTWC)  began  issuing  track  forecasts 
through  120h  in  2003.  Since  a  tropical  cyclone  could 
easily  form  and  intensify  to  a  powerful  storm  within  this 
120-h  window,  an  increased  need  exists  for  accurate 

prediction  of  tropical  cyclone  formation  from  operational 
global  models.  While  it  is  important  to  establish  the 
accuracy  of  forecasts  of  tropical  cyclone  formation  in 

global  models,  it  is  also  important  to  identify  and 
understand  factors  that  distinguish  forecasts  of  vortices 
that  are  correctly  forecast  to  intensify  into  tropical 

cyclones  (developers)  from  forecasts  of  vortices  that  are 
forecast  to  intensify  into  tropical  cyclones,  but  do  not 
(false  alarms) . 

A.  DEFINITION  OF  FORMATION 

The  criteria  for  defining  tropical  cyclone  formation 
are  not  universally  accepted,  although  all  definitions 
require  that  a  tropical  cyclone  be  convective  and  non- 
frontal,  originate  over  tropical  waters,  have  a  cyclonic 
surface  circulation,  and  have  a  warm  core  with  winds  that 
are  strongest  at  the  surface  and  decrease  with  height. 
Elsberry  (2003)  defines  formation  to  have  occurred  when 
there  exists  "a  non-frontal,  cyclonic  circulation  in  the 
tropics  that  is  closed  (ground-relative  westerly  winds  on 
the  equatorward  side  for  a  westward  moving  storm)  with 
maximum  10-minute  averaged  surface  sustained  winds  of  at 
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least  25  kt  (12.5  m  s”1)  that  is  accompanied  by  deep 
(throughout  most  of  the  troposphere)  convection  and  a 
radius  of  maximum  winds  such  that  the  Rossby  number  is  at 
least  one."  A  Rossby  number  of  at  least  one  would  require 
a  radius  of  maximum  wind  less  than  500  km  for  a  maximum 
wind  of  12.5  m  s_1  at  10°  latitude.  This  criterion  is 
included  to  distinguish  between  a  tropical  cyclone,  which 
has  maximum  winds  near  the  center,  and  a  monsoon  depression 
that  has  maximum  winds  at  a  larger  radius. 

Davis  and  Bosart  (2003)  use  the  requirement  that  the 
vortex  be  capable  of  self-amplification  through  air-sea 
interaction.  This  occurs  around  the  time  that  the  cyclone 
reaches  wind  speeds  of  34  kt  (17  m  s_1)  at  which  point  the 
character  of  the  air-sea  interaction  changes  such  that 
there  are  enhanced  fluxes  of  heat  and  moisture  between  the 
ocean  and  atmosphere.  This  is  the  typical  'storm' 

threshold.  Most  operational  centers,  including  the  NHC  and 
JTWC  begin  issuing  advisories  when  the  vortex  meets  the 
'depression'  threshold  of  25  kt  (12  m  s-1)  .  For  the  purpose 
of  this  study,  formation  will  be  considered  to  have 
occurred  at  the  time  of  the  first  advisory  issued  by  the 
NHC  or  JTWC,  which  is  also  the  time  of  its  first  entry  in 
the  'best-track'  database. 

B.  FAVORABLE  ENVIRONMENTAL  CONDITIONS  FOR  TROPICAL 

CYCLONE  FORMATION 

Gray  (1975)  cited  six  physical  parameters  that  create 
a  favorable,  though  not  sufficient,  environment  for  the 
formation  of  tropical  cyclones.  They  are:  (i)  large  values 
of  low-level  relative  vorticity;  (ii)  a  location  of  at 
least  500  km  from  the  equator  (to  gain  some  contribution 
from  Coriolis  force) ;  (iii)  weak  vertical  shear  of  the 

horizontal  winds;  (iv)  sea-surface  temperatures  exceeding 
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26°C  through  a  deep  oceanic  layer;  (v)  conditional 
instability  through  a  deep  atmospheric  layer;  and  (vi)  a 
moist  lower-  and  mid-troposphere.  The  first  three  describe 
dynamical  aspects  of  the  environment  while  the  last  three 
relate  to  its  thermodynamic  state. 

DeMaria  et  al .  (2001)  concluded  that  tropical  cyclone 

formation  in  the  Atlantic  is  constrained  by  vertical 
instability  and  midlevel  moisture  during  the  early  part  of 
the  hurricane  season,  and  by  vertical  wind  shear  in  the 
latter  part  of  the  season.  Molinari  et  al .  (2000) 

demonstrated  that  because  environmental  conditions  in  the 
eastern  North  Pacific  are  almost  always  favorable  for 
formation,  it  is  the  amplitude  of  approaching  tropical 
waves  that  is  the  most  important  factor  determining  whether 
or  not  tropical  cyclones  will  form. 

C .  TROPICAL  WAVES 

The  most  common  pre-existing  disturbance,  or  formation 
seedling,  in  both  the  Atlantic  and  eastern  North  Pacific  is 
the  tropical,  or  easterly,  wave  (Figure  1)  .  A  westward- 
moving  wave  train  develops  in  the  trade  winds  and 

convergence  on  the  eastern  side  of  the  trough  leads  to 
convection.  The  tropical  wave  that  affects  these  regions 
is  the  African  Easterly  Wave,  which  develops  from 
baroclinic-barotropic  instability  in  the  mid-level  easterly 
jet  over  Africa.  In  the  Atlantic,  40%  of  hurricanes  and 
65%  of  major  hurricanes  originate  as  tropical  waves  in  the 

area  between  North  Africa  and  the  Caribbean  (DeMaria  et  al . 

2001)  .  Avila  and  Pasch  (1995)  state  that  almost  all 
tropical  cyclones  in  the  eastern  North  Pacific  can  be 
traced  to  an  African  Easterly  Wave. 
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Figure  1  A  schematic  of  a  wave  in  the  easterly  trade 
winds  (From:  NASA  Earth  Observatory  Library 
http : / /earthobservatory . nasa . gov/Library/Hurri 
canes ;  June  2006) . 

D .  FORMATION  MECHANISMS 

While  the  environmental  conditions  required  for 
formation  are  understood,  and  the  importance  of  a  pre¬ 
existing  disturbance  to  organize  convection  and  concentrate 
potential  vorticity  (Davis  and  Bosart  2001)  is  recognized, 
the  processes  that  cause  this  disturbance  to  develop  into  a 
tropical  cyclone  are  not  well  understood.  Enhanced  global 
models  and  higher  resolution  satellite  imagery  have 
improved  the  forecast  process  significantly,  but 
insufficient  "in-situ"  observations  have  been  available 
over  the  tropical  oceans  to  verify  hypotheses  and  confirm  a 
single  mechanism  of  formation. 

In  general,  if  the  favorable  environmental  conditions 
are  met,  an  area  of  convection  experiences  positive 
feedback  in  that  warm  air  rises,  condenses  and  releases 
latent  heat.  That  heat  further  increases  the  buoyancy  of 
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the  air.  In  response  to  warming  in  the  rising  air,  the 
surface  pressure  falls  and  leads  to  a  rapid  inflow  of  air, 
which,  in  turn,  leads  to  more  thunderstorms.  Coriolis 
force  will  eventually  cause  the  low-level  inflow  to  develop 
a  cyclonic  circulation.  Warming  of  the  rising  air  causes 
air  pressure  to  rise  at  the  top  of  the  storm.  An  anti- 
cyclonic  outflow  develops  that  provides  an  exhaust  for  the 
storm  and  allows  the  above  sequence  of  events  to  continue. 

E.  USE  OF  GLOBAL  MODELS  TO  FORECAST  FORMATION 

As  described  by  Gray  (1975)  both  dynamical  and 
thermodynamic  conditions  may  create  a  favorable  environment 
for  tropical  cyclone  formation.  The  favorable 
thermodynamic  conditions  exist  over  broad  geographic 
regions,  change  slowly,  and  are  fairly  well  resolved  by 
global  models.  However,  the  dynamical  conditions  vary  on  a 
much  smaller  temporal  and  spatial  scales.  Not 
surprisingly,  it  is  therefore  much  more  difficult  for 
global  models  to  resolve  changes  in  these  variables. 

Due  to  these  limitations,  some  studies  (e.g.,  Beven 
1999)  concluded  that  the  global  operational  forecast  models 
had  no  significant  skill  in  forecasting  tropical  cyclone 
formation.  Improved  horizontal  and  vertical  resolution  in 
the  models  as  well  as  remotely  sensed  observations  has 
helped  global  models  better  resolve  the  physical  processes 
on  smaller  and  smaller  scales.  However,  Hennon  and  Hobgood 
(2003)  suggest  that  global  models  could  provide  skillful 
formation  forecasts  even  if  complex  mesoscale  processes  are 
neglected.  They  found  several  characteristics  associated 
with  the  large-scale  environment  over  the  North  Atlantic 
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that  may  provide  significant  measures  to  discriminate 
between  cloud  clusters  that  develop  into  tropical  cyclones 
and  those  that  do  not  develop. 

F.  A  GENESIS  PARAMETER 

Gray  (1975)  used  the  product  of  his  dynamic  criteria 
(low-level  relative  vorticity,  distance  from  the  equator, 
and  vertical  shear  of  the  horizontal  winds)  and  the  product 
of  the  thermodynamic  criteria  (sea-surface  temperature, 
conditional  instability  and  lower-  and  mid-tropospheric 
moisture)  to  define  a  Seasonal  Genesis  Parameter.  DeMaria 
et  al .  (2001)  developed  a  similar  tropical  cyclone  genesis 
parameter  for  the  Atlantic  that  was  the  product  of  vertical 
shear,  vertical  instability,  and  midlevel  moisture 
variables . 

Using  eight  predictors  from  large-scale  analyses, 
Hennon  and  Hobgood  (2003)  developed  a  probabilistic 
prediction  system  for  tropical  cyclone  formation.  They 
manually  tracked  cloud  clusters  in  satellite  imagery  and 
then  submitted  the  eight  variables  associated  with  each 
cluster  to  a  discriminant  analysis.  The  resulting  linear 
combination  of  the  variables  best  separates  cloud  clusters 
that  will  develop  into  tropical  depressions  from  those  that 
will  not. 

G.  PLAN  FOR  THE  THESIS 

In  this  thesis,  a  discriminant  analysis  similar  to 
that  used  by  Hennon  and  Hobgood  (2003)  is  applied  to 
forecasts  of  tropical  vortices  produced  by  several 
operational  global  models.  Vortices  are  identified  and 
tracked  based  on  their  analyzed  and  forecast  vorticity  as 
opposed  to  the  Hennon  and  Hobgood  approach  of  identifying 
candidate  cloud  clusters  in  satellite  imagery.  A 
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probability-based  model  is  used  to  define  the  likelihood 
that  a  tropical  vortex  that  is  forecast  by  the  operational 
model  to  become  a  tropical  cyclone  will  actually  develop 
into  a  tropical  cyclone.  Whereas  Hennon  and  Hobgood  (2003) 
used  the  discriminant  analysis  approach  for  the  broader 
problem  of  distinguishing  between  developing  and  non¬ 
developing  cloud  clusters,  this  study  is  limited  to  cases 
in  which  a  vortex  is  forecast  to  develop  into  a  tropical 
cyclone,  and  the  objective  is  to  provide  the  probability 
that  the  model  forecast  is  true  or  false. 

The  Vortex  Tracking  (VORTRACK)  system  of  Harr  (2006) 
is  the  key  to  this  type  of  data  analysis.  Therefore,  the 
first  part  of  Chapter  II  (Methodology)  will  describe  the 
VORTRACK  system.  The  data  analysis  approach  will  then  be 
summarized.  For  each  operational  global  model  and  forecast 
interval  considered,  the  approach  included  producing 
scatterplots ,  probability  distribution  functions  (PDFs),  a 
linear  discriminant  analysis  (which  included  a  jackknife 
routine  for  application  to  independent  data) ,  and 
performance  assessment  to  include  the  Heidke  Skill  Score 
(HSS)  and  Receiver  Operating  Characteristics  (ROC)  curves. 

In  Chapter  III,  the  results  will  be  presented,  which 
will  include  the  parameters  used  in  each  discriminant 
function  along  with  relevant  PDFs,  as  well  as  the  HSS  and 
ROC  curve  for  each  discriminant  function.  Chapter  IV  will 
summarize  the  conclusions,  highlight  some  of  the  challenges 
posed  by  this  type  of  data  analysis,  and  make 
recommendations  for  further  study. 
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II .  METHODOLOGY 


To  discriminate  between  operational  model  forecasts  of 
developing  and  non-developing  vortices,  it  is  necessary  to 
extract  parameters  that  are  relevant  to  tropical  cyclone 
formation  relative  to  each  tropical  vortex.  These 
parameters  must  be  identified  in  forecasts  of  varying 
lengths  for  comparison  with  the  parameters  in  the  verifying 
analyses.  Furthermore,  it  is  important  to  capture  these 
characteristics  among  several  operational  models.  The 
database  defined  by  the  VORTRACK  system,  which  is 
summarized  in  Figure  2,  is  used  to  identify  forecast  model 
parameters  that  have  the  most  predictive  value  with  regards 
to  formation. 

The  principal  objective  of  this  research  is  to  create 
such  a  tool  that  can  discriminate  between  a  developer  and  a 
false  alarm  in  the  global  model  forecasts.  This  tool  will 
be  based  on  a  probabilistic  assessment  derived  from  a 
linear  discriminant  analysis. 

A.  VORTRACK 

The  VORTRACK  application  was  developed  at  the  Naval 
Postgraduate  School  (NPS)  so  that  output  from  numerical 
models  could  more  easily  be  included  in  the  process  of 
forecasting  the  intensification  of  a  tropical  low  into  a 
tropical  storm.  A  prototype  was  run  for  the  2005  hurricane 
season  at  both  NPS  and  the  NHC .  There  are  two  primary 
components  of  VORTRACK.  The  first  is  the  data  processing 
segment  which  is  the  Tropical  Cyclone  Vortex  Tracking 
Program  (TCVTP) .  This  program  outputs  the  tracks  of  all 
eligible  tropical  vortices  as  well  as  the  corresponding 
values  of  14  environmental  parameters.  The  resulting  ASCII 
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text  file  is  stored  in  one  location  if  the  vortex  can  be 
matched  in  space  and  time  with  a  vortex  in  the  verifying 
analysis  field,  and  in  another  location  if  the  vortex 
appears  only  the  forecast  fields.  The  second  component  of 
VORTACK  is  a  web-interface  output  segment  (Harr  2006) . 
Vortex  locations  and  environmental  parameter 

characterizations  from  the  TCVTP-generated  ASCII  text  files 
were  used  for  this  thesis. 


I  Forecast  Model  Analyses  and  Forecast  Fields 

r~ i 


Figure  2  Schematic  of  the  TCVTP  component  of  VORTRACK 
(From:  Harr  2006) . 


1 .  Tropical  Cyclone  Vortex  Tracking  Program  (TCVTP) 

The  objective  tropical  vortex  identification  and 
tracking  technique,  TCVTP,  was  utilized  to  examine  analyzed 
and  forecast  fields  of  three  operational  global  numerical 
models  during  the  2005  hurricane  season  in  the  North 
Atlantic.  The  models  that  had  been  catalogued  in  the  2005 
database  include  the  Navy  Operational  Global  Atmospheric 
Prediction  System  (NOGAPS) ,  the  National  Centers  for 
Environmental  Prediction's  Global  Forecast  System  (GFS) , 
and  the  United  Kingdom  Meteorological  Office  (UKMET)  global 
model . 
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For  each  tropical  vortex,  which  is  defined  by  the 
850  hPa  relative  vorticity  above  a  threshold  value  of 
1.0  x  10“5  s”1,  14  environmental  parameters  in  model  analyses 
and  forecasts  that  are  relevant  to  tropical  cyclone 
formation  (Table  1)  are  catalogued  in  the  TCVTP.  In  this 
study,  only  nine  of  the  14  parameters  were  used.  Those 
parameters  shaded  on  Table  1  were  not  used  in  this  study. 
For  example,  the  850-500  hPa  average  relative  vorticity  was 
chosen  over  the  850  hPa  relative  vorticity.  The  700- 
400  hPa,  700-300  hPa  warm  core,  and  precipitation 
parameters  were  not  used  because  these  values  were  not 
available  for  the  UKMET  model  and  therefore  they  could  not 
be  used  for  model  inter-comparison. 

Table  1  Analyzed  and  forecast  quantities  used  to 
identify  physical  characteristics  associated 
with  each  tropical  vortex.  Warm  core 
measurements  are  defined  as  a  temperature 
difference  between  the  vortex  and  the 
environment  (From:  Harr  2006)  . 


850  hPa  Relative  Vorticity 

Sea- level  pressure 

Minimum (mb) 

Shallow  Vertical  Wind  Shear 

(850-500  hPa) 

Deep  Vertical  Wind  Shear 

(850-200  hPa) 

850-200  hPa  Geopotential 

Height  Thickness 

700-500  hPa  Warm  Core 

Vertical  motion  at  500  hPa 

700-400  hPa  Warm  Core 

700-500  hPa  Vapor  Pressure 

700-300  hPa  Warm  Core 

850-500  hPa  Average  Relative 

Vorticity 

Sea-level  Pressure  Difference 

between  the  Vortex  and  the 

Environment 

Total  Precipitation 

Convective  Precipitation 
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B.  SOME  DEFINITIONS 

The  characterization  of  a  vortex  as  developer  or  non¬ 
developer  was  based  on  the  850-500  hPA  average  relative 
vorticity  and  700-500  hPa  warm  core  values  as  compared  to 
threshold  values  for  these  parameters.  The  threshold  value 
(PT)  for  each  parameter  is  defined  as  the  lowest  tercile  of 
that  parameter  averaged  for  all  developers  at  the  time  of 
formation.  Time  of  formation  (To)  is  defined  as  the  time 
that  the  tropical  cyclone  was  first  entered  in  the  best- 
track  database.  A  developer  is  any  tropical  vortex  that 
was  eventually  entered  in  the  best-track  database.  For  the 
purpose  of  this  study,  this  included  all  forecast  hits 
(vortex  was  forecast  to  exceed  the  threshold  values  and  the 
corresponding  vortex  in  the  verifying  analysis  was  a 
developer)  and  misses  (vortex  was  not  forecast  to  exceed 
the  threshold  values  but  was  in  fact  a  developer) . 

A  non-developer  is  defined  as  a  vortex  tracked  in  the 
TCVTP  database  that  never  entered  the  NHC  best-track 
database,  which  may  include  three  sub-categories.  A  false 
alarm  is  a  tropical  vortex  that  was  forecast  to  exceed  both 
threshold  values,  but  is  actually  a  non-developer  because 
it  did  not  verify  above  those  threshold  values.  The 
majority  of  vortices  in  the  TCVTP  database  was  not  forecast 
to  exceed  both  threshold  values  and  did  not  verify  above 
the  threshold  values;  these  correct  nulls  were  not 
considered  in  this  study.  The  third  category  consists  of 
vortices  in  the  TCVTP  that  were  forecast  to  be,  and 
according  to  model  analyses  verified  above,  both  threshold 
values  but  were  never  entered  in  the  NHC  best-track 
database.  In  these  cases,  there  were  other  parameters  not 
considered  in  this  study  (e.g.,  low-level  wind  shear)  that 
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must  have  prevented  these  vortices  from  becoming 
developers.  Even  though  they  meet  the  definition  of  non¬ 
developers,  since  the  forecast  verified  in  the  correct 
category  (i.e.,  upper-right  quadrant  of  the  scatterplots  in 
Appendix  A)  ,  they  are  not  regarded  as  false  alarms  and  were 
also  not  considered  in  this  study. 

C .  PRELIMINARY  ANALYSIS 

Since  by  definition  any  tropical  cyclone  must  have 
positive  lower-tropospheric  relative  vorticity  and  a  warm 
core,  all  tropical  vortices  in  the  TCVTP  database  for  each 
model  at  each  forecast  interval  were  plotted  on  a  graph 
with  850-500  hPa  average  relative  vorticity  on  the  x-axis 
and  700-500  hPa  warm  core  on  the  y-axis.  Figure  3  is  an 
example  of  this  scatterplot  for  the  GFS  12-h  model  data. 
Two  clusters  were  expected:  developers  with  a  centroid  in 
the  upper-right  quadrant,  and  non-developers  with  a 
centroid  in  the  lower-left  quadrant.  As  evident  on 
Figure  3,  such  a  separation  between  developers  and  non¬ 
developers  in  the  model  forecasts  was  not  observed. 

This  and  scatterplots  at  longer  forecast  intervals 
demonstrated  that  the  GFS  model  does  not  have  skill  in 
distinguishing  between  developers  and  non-developers  when 
considering  the  two  most  basic  prerequisites  of  tropical 
cyclones  (positive  relative  vorticity  and  a  warm  core) . 
Similar  results  were  achieved  with  the  NOGAPS  and  UKMET 
models  at  all  forecast  periods.  As  a  result,  the  aim  of 
this  study  was  shifted  to  discrimination  between  developers 
and  false  alarms.  Since  a  false  alarm  is  a  non-developing 
vortex  with  forecast  values  of  850-500  hPa  average  relative 
vorticity  and  700-500  hPa  warm  core  above  threshold  values, 
these  false  alarms  also  are  in  the  upper-right  quadrant  on 
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Figure  3.  That  is,  only  developers  entered  in  the  best- 
track  data  and  non-developers  in  the  upper-right  quadrant 
of  the  vorticity  /  warm  core  scatterplots  are  considered  in 
the  following  sections. 


GFS  1 2-h  Forecast  Avg.  850-500  hPa  Vorticity  and  700-500  hPa  Warm  Core 


Figure  3  A  scatterplot  of  850-500  hPa  average  relative 
vorticity  and  700-500  hPa  warm  core  for  all 
vortices  in  the  TCVTP  database  for  the  GFS 
model  at  the  12-h  forecast  time.  The  dark 
solid  lines  represent  the  calculated  threshold 
values  (for  this  model  and  forecast  period) 
for  these  two  parameters . 

For  this  GFS  12-h  example,  the  vortices  of  interest 
are  identified  in  Figure  4.  On  this  figure,  black  squares 
represent  developers.  Solid  circles  represent  false 

alarms.  Cyan  represents  vortices  that  were  false  alarms  in 
warm  core  only  (i.e.,  verified  above  the  vorticity 
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threshold) .  Magenta  represents  vortices  that  were  false 
alarms  in  vorticity  only  (i.e.  verified  above  the  warm  core 
threshold) .  Green  represents  vortices  that  were  false 
alarms  in  both  warm  core  and  vorticity.  The  objective  of 
the  linear  discriminant  analysis  is  then  to  distinguish 
between  the  developers  (squares  on  Figure  4)  and  false 
alarms  (solid  colored  circles  on  Figure  4) . 


GFS  1 2-h  Forecast  Avg.  850-500  hPa  Vorticity  and  700-500  hPa  Warm  Core 
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As  in  Figure  3  except  with  non  developers f 
developers  and  three  kinds  of  false  alarms 
annotated  as  indicated  in  the  inset. 


The  next  step  in  the  data  analysis  was  to  include 
additional  parameters  on  these  scatterplots .  In  Figure  5, 
vortices  are  plotted  as  in  Figure  4,  except  that  vortices 
with  500  hPa  omega  values  above  the  threshold  value  for 
that  parameter  are  indicated  (with  a  blue  versus  black 
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square  in  the  case  of  developers  and  with  a  triangle  versus 


circle  in  the  case  of  false  alarms)  .  The  few  open  red 
circles  in  the  upper  right  quadrant  represent  vortices  that 
verified  above  the  thresholds  for  both  warm  core  and 
vorticity  and  were  therefore  not  considered  false  alarms. 
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As  in  Figure  4,  except  with  vortices  exceeding 
the  threshold  for  the  500  hPa  vertical  motion 
(omega)  parameter  annotated  as  indicated  in 
the  inset. 


The  goal  in  completing  these  scatterplots  was  to 
determine  which  parameters  had  the  best  potential  for 
discriminating  between  developers  and  false  alarms  and  thus 
to  reduce  the  number  of  parameters  considered  in  the 
discriminant  analysis.  However,  inspection  of  the 
scatterplots  revealed  no  systematic  tendencies  associated 
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with  various  parameters  that  distinguished  between 
developers  and  false  alarms.  Therefore  PDFs  of  each 
parameter  and  vortex  category  were  considered  and  are 
described  in  the  next  sub-section.  For  information 
purposes,  the  scatterplots  including  each  of  the  seven 
parameters  in  the  GFS  model  at  12h  are  displayed  in 
Appendix  A,  and  the  scatterplots  including  omega  in  all 
three  models  at  all  ten  forecast  intervals  are  displayed  in 
Appendix  B.  The  scatterplots  in  Appendix  B  are  useful  for 
illustrating  the  number  of  cases  considered  in  each  test, 
which  helps  account  for  some  of  the  problems  encountered  in 
the  data  analysis  at  the  longer  forecast  intervals. 

D.  PROBABILITY  DISTRIBUTION  FUNCTIONS  (PDFS) 

Probability  Density  Functions  (PDFs)  were  completed 
for  the  nine  parameters  in  each  model  at  each  forecast 
interval  and  are  contained  in  Appendix  C.  An  example  from 
the  UKMET  12-h  forecast  data  is  provided  in  Figure  6.  The 
goal  in  completing  the  PDFs  was  to  reduce  the  number  of 
parameters  to  be  considered  in  the  discriminant  analysis. 
It  was  assumed  that  those  parameters  that  had  the  smallest 
amount  of  overlap  in  the  blue  and  red  curves  would  be  the 
best  discriminators  between  developers  and  false  alarms. 
Whereas  this  may  be  true  when  considering  an  individual 
parameter  it  became  evident  in  preliminary  linear 
discriminant  analyses  that  the  combinations  of  parameters 
behaved  in  ways  that  were  not  obvious  from  the  individual 
plots  (e.g..  Figure  6)  .  Thus,  a  more  comprehensive 
analysis  of  all  combinations  of  parameters  needed  to  be 
conducted . 
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UKM:  12-h  Parameter  Distributions 
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Figure  6  Probability  Distribution  Functions  of  non¬ 
developers  in  green,  developers  in  blue,  and 
false  alarms  in  red  for  the  nine  parameters 
derived  from  the  UKMET  12-h  forecast  relative 
to  tropical  vortices  in  the  TCVTP. 


E.  LINEAR  DISCRIMINANT  ANALYSIS 

The  goal  of  a  linear  discriminant  analysis  is  to 

discern  group  membership  based  on  a  vector  (x)  having  k 
attributes  that  are  observed  for  n  cases  (Wilks,  2005)  .  A 

training  sample  that  contains  n  cases  is  used  to  define  the 
discriminant  function.  The  number  of  groups  (g)  is 
specified  a  priori  and  each  case  must  belong  to  one  and 

only  one  group.  Group  membership  must  be  specified  for  all 
cases  in  the  training  sample.  In  this  study,  g=2  since 
developers  and  false  alarms  are  the  two  groups,  and  k=9 
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since  nine  parameters  from  the  TCVTP  database  are 
considered.  The  training  sample  size  (n)  varies  by  model 
and  forecast  interval. 

The  discriminant  analysis  process  consists  of  two 
segments.  Discrimination  is  the  process  of  using  the 
training  data  to  find  a  function  of  the  attributes  that 
best  discriminates  between  the  groups  (the  discriminant 
function) .  Classification  is  the  process  of  assigning  new 
cases  to  one  of  the  groups  using  the  discriminant  function. 
Alternately,  classification  may  involve  estimating  the 
probabilities  of  group  membership.  In  this  study,  both 
methods  of  classification  were  also  applied  to  independent 
data  using  a  'jackknife'  technique  to  test  the  skill  of  the 
discriminant  function.  This  process  will  be  described  in  a 
later  section. 

A  discriminant  function  is  needed  for  each  model  and 
forecast  interval  (30  in  total)  .  For  a  given  model  and 

forecast  interval,  the  data  matrix  (n  x  k)  can  be  divided 

into  the  two  groups  that  result  in  two  matrices  with 
dimensions  (ni  x  k)  and  (ri2  x  k)  where  n  =  ni  +  n2 .  The 
number  of  developers  (ni)  and  false  alarms  (n2)  in  the  model 

data  varies  by  model  and  forecast  period.  Hence,  the  first 

matrix  has  dimensions  of  (ni  x  9)  and  the  second  matrix  has 

dimensions  (n2  x  9)  .  The  goal  is  to  find  a  linear  function 

of  the  k  parameters  or  the  linear  combination  aTx  that  will 
best  classify  the  group  membership  of  future  cases.  This 
linear  combination  is  defined  as  the  discriminant  function. 

Assuming  the  two  populations  corresponding  to  the 

groups  have  the  same  covariance  structure,  Wilks  (2005) 
summarizes  the  approach  taken  by  statistician  R.A.  Fisher 
as  finding  "the  vector  a  as  that  direction  in  the  k- 
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dimensional  space  of  the  data  that  maximizes  the  separation 
of  the  two  means,  in  standard  deviation  units,  when  the 
data  are  projected  onto  a."  This  procedure  requires  that  a 
maximizes 

(aTXj  -aTx2V 

^ - le  V  '  (1) 

aT  [Spool  > 

where  xi  and  X2  are  the  mean  vectors  for  each  group  and  Spooi 
is  the  estimated  common  covariance  matrix  of  the  two  groups 
which  is  a  weighted  average  of  the  two  sample  covariance 
matrices 


ni  -1 

iij  +n2  -2 


n2  -1 
n i  +n2  -2 


[sj- 


(2) 


The  advantage  of  finding  the  direction  a  is  that  the 
data  vector,  x,  can  be  transformed  into  a  scalar  variable, 
5i=aTx,  which  is  known  as  Fisher's  linear  discriminant 
function  as  defined  above.  The  groups  of  multivariate  data 
are  hence  transformed  to  groups  of  univariate  data 
distributed  along  the  a  axis.  The  vector  a,  which  lies  in 
the  direction  of  maximum  separation  is  given  by 


a  =  [Spool  ]  1  (X1  -*2)  ’ 


(3) 


and  Fisher's  linear  discriminant  function  becomes 

5j  =  aTx  =  (Xj  -  x2)T[spool]_1x  . 


(4) 


This  function  maximizes  the  scaled  distance  (D)  between  the 
two  sample  means  in  the  training  data  where  it  is  defined 
by: 


aT(Xi  - x2)  =  (Xj  - x2)T[Spool]“1(x1  - x2)  =  D2  . 


(5) 
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Future  cases  will  be  classified  as  belonging  to  either 
Group  1  or  Group  2  according  to  the  value  of  5i.  Because  5i 
is  a  dot  product,  it  is  a  scalar,  or  one-dimensional 
projection  of  the  vector  x  onto  the  direction  of  maximum 
separation,  a.  The  simplest  method  of  classification  is  to 
assign  an  individual  case  in  x,  to  Group  1  if  the 
projection  aTx  is  closer  to  the  projection  of  the  Group  1 
mean  and  assign  it  to  Group  2  if  it  is  closer  to  the 
projection  of  the  Group  2  mean. 

The  midpoint  between  the  means  of  the  two  groups  along 
the  a  axis  is  given  by: 

m  =  ^-(aTx1  +aTx2)  =  ^-aT(x1  +x2)  =  ^-(x1  -x2)T[Spool]_1(x1  +x2)  .  (6) 

Therefore  a  case  in  x  is  classified  according  to  the 
following  rule: 

Assign  to  Group  1  if  aTx  >  m 


Assign  to  Group  2  if  aTx  <  m  . 


Alternately,  a  probabilistic  classification  can  be 
done.  Using  this  method,  the  probability  that  a  case  in  x 
belongs  to  each  of  the  two  groups  is  determined  according 
to  Bayes'  Theorem: 


Pr{Groupl|x}  = - ^ 

1  '  j  P,f,(x) 


Pifi(x) 


+  P2f2(x) 


Pr  {Group  2|x}  = - PlfW - 

1  F  1  J  p1f1(x)  +  p2f2(x) 


(7) 


(8) 


In  Equations  (7)  and  (8),  the  pi  and  P2  values  are  the  prior 
probabilities  of  group  membership,  which  is  simply  the 
relative  frequency  with  which  group  1  and  group  2, 


respectively,  are  represented  in  the  training  data.  Thus  fi 
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and  f 2  are  the  PDFs  for  groups  1  and  2,  respectively.  If  it 
can  be  assumed  that  the  PDFs  are  multivariate  normal 
distributions.  Equations  (7)  and  (8)  become: 


f 

_1  .  1 
2exp[--(x  -  i1)T[Sir1(x  -  Xj)] 

Pi 

(sll 

V 

J 

( 

_1  j 

2  exp[-  -  (x  -  Xj  )T  [Sj  ]_1  (x  -  Xj )] 

( 

_1  j 

2  exp[--(x  -  x2)T[S2]_1(x  -  x2)] 

Pi 

IKI 

+  p2 

V 

J 

V 

) 

(9) 


(10) 


In  this  study,  the  probability  threshold  at  which  a 
case  was  classified  as  belonging  to  one  group  or  the  other 
was  incrementally  adjusted  between  0  and  1.  This  process 
is  explained  Chapter  II,  Section  G  which  describes  the  ROC 
curves . 

F.  JACKKNIFE  ROUTINE 

A  Jackknife  routine  eliminates  one  case  at  a  time  and 
recalculates  the  relevant  statistics  (Wilks  2005) .  For 
example,  the  first  case  can  be  removed  from  the  data  matrix 
which  results  in  a  matrix  with  dimensions  ([n-1]  x  k) .  The 
linear  discriminant  analysis  is  performed  on  this  matrix 
and  the  resulting  discriminant  function  can  then  be  used  to 
classify  the  excluded  case.  The  process  is  repeated  by  re¬ 


inserting 

the 

first  case  and  removing  the 

second 

case, 

and 

so  on . 

This 

routine  provides 

a  method 

for  testing 

the 

function 

with 

independent  data 

in  the  absence 

of  a 

new 

dataset  . 

G.  HEIDKE  SKILL  SCORE  (HSS) 

Forecast  verifications  are  often  placed  in  contingency 
tables  similar  to  Table  2. 
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Table  2 


A  contingency  table  in  which  forecast  status  is 
compared  with  observed  status.  Hits  are 
represented  by  a,  false  alarms  by  b,  misses  by 
c,  and  correct  nulls  by  d. 


Observed 

Yes 

Observed 

No 

Forecast 

Yes 

A 

b 

Forecast 

No 

c 

d 

According  the  Wilks  (2006) ,  the  reference  accuracy 
measure  in  the  HSS  is  the  proportion  of  random  forecasts 
(statistically  independent  of  the  observations)  that  would 
be  correct.  Perfect  forecasts  receive  a  score  of  1  and 
forecasts  equivalent  to  random  chance  receive  a  score  of  0 . 
If  the  forecasts  are  worse  than  random  forecasts,  they 
receive  a  negative  score.  The  probability  of  a  correct 
"yes"  forecast  by  chance  is  equal  to  the  probability  of  a 
"yes"  forecast  times  the  probability  of  a  "yes" 
observation,  or 

(a  +  b)  (a  +  c)  _  (a  +  b)(a  +  c)  (n 

n  n  n2 

and  the  probability  of  a  correct  "no"  forecast  is 

(b  +  d)(c  +  d)  (b  +  c)(c  +  d)  .  (12 

n  n  n2 


These  definitions  are  equivalent  to  the  assumption 
that  the  forecasts  and  observations  are  independent.  The 
HSS  is  then  equal  to  the  ratio  of  the  number  correct  to  the 
number  incorrect  by  chance,  divided  by  one  minus  the  number 
correct  by  chance,  which  simplifies  to: 


HSS 


_ 2(ad  -  be) _ 

(a  +  c)(c  +  d)  +  (a  +  b)(b  +  d) 


(13) 
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The  HSS  is  first  calculated  for  discriminant  functions 
containing  all  possible  combinations  of  the  nine 
parameters.  The  discriminant  function  that  shows  the  most 
skill  is  then  submitted  to  the  jackknife  procedure. 

H.  RECEIVER  OPERATING  CHARACTERISTICS  (ROC)  CURVES 

The  use  of  Receiver  Operating  Characteristic  (ROC) 
curves  originated  in  the  signal  processing  community.  In 
this  sense,  the  threshold  for  which  a  signal  is  determined 
to  be  present  is  incrementally  adjusted  between  the  point 
at  which  a  signal  is  always  detected  and  the  point  at  which 
a  signal  is  never  detected.  The  probability  of  detection, 
or  hit  rate  (equivalent  to  d/  (c+d)  in  Table  2),  and  the 
false-alarm  rate  (equivalent  to  b/ (a+b)  in  Table  2)  are 
calculated  for  each  of  these  thresholds.  These  points  are 
plotted  on  a  graph  with  the  false-alarm  rate  (0  to  1)  along 

the  x-axis  and  the  hit  rate  (0  to  1)  along  the  y-axis.  The 

resulting  points  form  the  ROC  curve. 

A  ROC  curve  that  followed  a  diagonal  from  (0,0)  to 
(1,1)  would  indicate  that  the  'receiver'  produced  as  many 
false  alarms  as  hits.  The  area  under  the  curve  in  this 
case  would  be  0.5.  The  ideal  ROC  curve  would  follow  along 
the  left  and  top  axes  which  would  indicate  that  the 
receiver  had  a  perfect  hit  rate  and  produced  no  false 
alarms.  The  area  under  this  curve  would  be  1.  In  reality, 
most  receivers  fall  somewhere  between  these  two  extremes. 
The  closer  a  ROC  curve  is  to  the  upper  left  corner,  the 

closer  the  receiver  is  to  ideal  (all  hits  and  no  false 

alarms)  .  Therefore,  the  area  under  the  curve  (AUC)  is  one 
measure  of  performance. 

For  purpose  of  this  study,  the  threshold  for  which  a 
candidate  vortex  was  classified  as  being  a  developer  was 


24 


varied.  By  default,  this  number  is  0.5.  If,  based  on  the 
jackknifed  discriminant  function,  a  vortex  has 
classification  scores  of  0.51  for  Group  1  (developers)  and 
0.49  for  Group  2  (false  alarms)  it  is  automatically 
classified  as  a  developer.  By  allowing  the  threshold  to 
vary,  some  degree  of  fuzziness  is  introduced  into  the 
analysis  and  an  additional  measure  of  skill,  the  Area  Under 
the  Curve  (AUC) ,  becomes  available.  Furthermore,  the 
classification  threshold  value  that  produces  the  point  on 
the  ROC  curve  that  is  farthest  from  the  diagonal  is  chosen 
as  the  optimal  threshold  for  the  applicable  model  and 
forecast  period. 
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Ill . RESULTS 


A.  PRESENTATION  OF  DISCRIMINANT  ANALYSIS  STATISTICS 

Table  3  is  divided  into  ten  blocks  for  the  ten 
forecast  intervals  with  a  row  for  each  of  the  three  models. 
The  first  and  second  columns  list  the  model  and  forecast 
period  (tau) ,  respectively.  The  third  column  lists  the 
predictors  that  were  used  in  the  applicable  discriminant 
function  (Table  4  lists  the  parameters  that  corresponds  to 
the  numbers  in  this  column) .  The  fourth  and  fifth  columns 
contain  two  HSS  values.  The  first  score  is  for  the 
discriminant  function  using  dependent  data  and  the  second 
score  is  for  the  discriminant  function  using  the 
independent  data.  Both  skill  scores  were  calculated  for  a 
classification  threshold  of  0.5.  The  sixth  column  gives 
the  Area  Under  the  Curve  (AUC)  for  the  ROC  curves  displayed 
in  the  subsequent  figures. 

The  next  section  includes  a  summary  of  the  results  for 
each  forecast  interval.  One  of  the  key  results  is  the 
parameters  that  are  common  to  the  discriminant  functions 
applied  to  each  model.  The  relevant  HSS  and  ROC  curves  are 
also  summarized.  In  the  summary  of  12-h  forecasts,  a 
detailed  example  of  the  methodology  as  applied  to  the  UKMET 
12-h  model  data  will  be  provided.  The  last  section  of  this 
chapter  will  describe  two  classification  case  studies. 
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Table  3 


Results  of  the  Linear  Discriminant  Analysis. 


Predictors  Used 
1,  2 ,  4 ,  5 
1,  2,  3,  5 ,  8,  9 

1,  2,  3,  6,  7,  8 , 


Maximum  HSS 
0.5924 
0.6063 
0.7856 


Jackknife  HSS 
0.2917 
0.2475 
0.5470 


AUC 

0.5382 

0.5500 

0.7289 


1,  2,  3 ,  4 ,  5 ,  6, 
_ 7 ,  8 _ 

2 , 3 , 5 , 8 ,  9 

2, 3, 6, 7, 8, 9 


0.5259 

0.6528 


0.2582 

0.4215 


0.5974 

0.6447 


1,  2,  4,  6,  7 ,  8 ,  9  0.5086 

1, 2, 4f  5, 6,  9 _ 0.5816 

_ 2,3,5,  6 _ 0.6252 


0.1872 

0.2763 

0.4250 


0.569 

0.5853 

0.6000 


1,  2,  3,  8 
2  r  5,  7,  8,  £ 
1/2, 4, 6, 7, 9 


0.4811 

0.6476 

0.7952 


0.3456 

0.3393 

0.5597 


0.6033 

0.5500 

0.7500 


1/  7,  8 
1,3, 5, 7, 8, 9 
1, 4, 5, 6, 7,  9 


0.1261 

0.3902 

0.9344 


0.1614 

0.0691 

0.4516 


0.5500 

0.5136 

0.7250 


1,  2,  3,  4,  5,  7,  9 _ 0.4167 

1,2, 4, 6, 7, 9  0.6704 

_ 2,  3,  4 _ 0.6102 


0.1619 

0.3987 

0.3060 


0.5611 

0.6000 

0.5800 


2,  4,  6,  7,  9 
1,2, 3, 4, 5, 7 
1,2, 7, 8 


0.5098 

0.7492 

0.8095 


0.2735 

0.3194 

0.4783 


0.6375 

0.6409 

0.6643 


2  r  3,  6, 

4,6,* 

5 


0.5000 

0.3241 

1.0000 


0.3750 

0.2118 

0.0000 


0.6167 

0.5500 

0.0000 


2,3,4 

1 

lr  2,  9 


0.6038 

0.0000 

1.0000 


0.3123 

-0.1030 

0.3077 


0.5800 

0.4300 

0.6667 


1,  2,  3,  5,  6,  7 
1 

_ 2,3, 6 _ 


0.5503 

0.0000 

0.7200 


0.0979 

-0.0593 

0.4235 


0.5500 

0.3500 

0.6500 


Table  4  List  of  parameters  corresponding  to  numbers  in 
column  3  of  Table  3. 


1 

2 

3 

4 

5 

6 

7 

8 

9 

850-500 

700- 

850- 

850- 

850- 

500 

700-500 

SLP 

SLP 

hPa 

500 

500 

200 

200 

hPa 

hPa 

Diff¬ 

Min¬ 

Average 

hPa 

hPa 

hPa 

hPa 

Omega 

Vapor 

erence 

imum 

Vorticity 

Warm 

Shear 

Shear 

Thick¬ 

Pressure 

Core 

ness 

B.  DISCUSSION  OF  RESULTS 
1.  12-h  Forecasts 

a.  Application  of  Methodology  to  UKMET  Forecast 

A  total  of  19  developers  and  31  false  alarms  were 
identified  in  the  scatterplot  for  the  UKMET  12-h  model  data 
(see  Appendix  B)  .  The  pertinent  matrices  for  developers 
(19  x  9)  and  false  alarms  (31  x  9)  were  submitted  to  a 
series  of  linear  discriminant  analyses  that  considered  all 
possible  combinations  of  the  nine  parameters.  Using  a 
contingency  table  (as  in  Table  2),  the  HSS  was  calculated 
for  each  of  the  discriminant  functions  and  the  function 
that  resulted  in  the  highest  HSS  was  selected.  In  the  UKMET 
12-h  case,  a  discriminant  function  using  seven  of  the  nine 
parameters  (850-500  hPa  Average  Vorticity,  700-500  hPa  Warm 
Core,  850-500  hPa  Shear,  500  hPa  Omega,  700-500  hPa  Vapor 
Pressure,  SLP  Difference,  and  SLP  Minimum)  exhibited  the 
most  skill . 

Table  5  indicates  that  only  five  cases  were 
misclassif ied  by  the  discriminant  function  applied  to  the 
UKMET  48-h  forecast.  Three  cases  did  develop  despite  being 
classified  as  false  alarms  and  two  cases  did  not  develop 
despite  being  classified  as  developers.  These  results,  as 
applied  to  the  calculation  of  the  HSS,  are  more  simply 
displayed  in  a  contingency  table  (Table  6)  ,  which  can  also 
be  used  to  compute  hit  and  false-alarm  rates  for  use  in  a 
ROC  diagram. 
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The  next  step  was  to  perform  a  jackknife  routine 
on  the  UKMET  12-h  forecast  data  and  recalculate  all 
relevant  statistics.  As  expected,  the  contingency  table 
for  the  discriminant  function  using  the  independent  data 
(Table  7)  has  less  skill  (HSS=0.55)  compared  to  that  using 
dependent  data  (HSS=0.79) . 


Table  5  Classification  results  for  the  relevant 
discriminant  function  as  applied  to  dependent 
data  from  the  UKMET  12-h  model.  Column  one 
lists  the  actual  group  membership  of  each 
vortex;  a  '1'  represents  a  developer  and  a  '2' 
represents  a  false  alarm.  Column  two  lists  the 
group  to  which  each  vortex  was  assigned  by  the 
discriminant  function.  The  probabilities, 

according  to  the  discriminant  function,  that 
the  vortex  belongs  to  group  '1'  and  '2'  are 
listed  in  columns  three  and  four,  respectively. 
Shaded  rows  represent  vortices  that  were 
misclassif ied  by  the  discriminant  function. 
The  three  light  grey  rows  represent  vortices 
that  did  develop,  but  were  classified  as  false 
alarms .  The  two  dark  grey  rows  represent 

vortices  that  did  not  develop,  but  were 

classified  as  developers. 


Observed 

Forecast 

Group  1 
Probability 

Group  2 
Probability 

1 

1 

0.9504 

0.0496 

1 

1 

0.6554 

0.3446 

1 

1 

0 . 6665 

0.3335 

1 

1 

0.9802 

0.0198 

1 

1 

0 . 6624 

0.3376 

1 

1 

0.5493 

0 .4507 

1 

2 

0.2386 

0 .7614 

1 

1 

0.9644 

0 . 0356 

1 

1 

0.9048 

0.0952 

1 

2 

0.1819 

0.8181 

1 

2 

0.2827 

0.7173 

1 

1 

0.6362 

0.3638 

1 

1 

0.9287 

0 . 0713 

1 

1 

0.5582 

0 .4418 

1 

1 

0.8837 

0.1163 

1 

1 

0 . 9443 

0 . 0557 
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Observed 

Forecast 

Group  1 
Probability 

Group  2 
Probability 

1 

1 

0.9136 

0.0864 

1 

1 

0 . 8541 

0 .1459 

1 

1 

0 . 9724 

0.0276 

2 

2 

0 . 4337 

0.5663 

2 

1 

0 . 9111 

0.0889 

2 

2 

0.0458 

0.9542 

2 

2 

0.1063 

0 .8937 

2 

2 

0.0131 

0.9869 

2 

2 

0 . 0027 

0.9973 

2 

2 

0.0257 

0 . 9743 

2 

2 

0.1172 

0 . 8828 

2 

2 

0 . 0228 

0.9772 

2 

2 

0.0104 

0.9896 

2 

2 

0.2150 

0.7850 

2 

2 

0.1772 

0 . 8228 

2 

2 

0.0466 

0.9534 

2 

2 

0.0409 

0.9591 

2 

2 

0 .1677 

0.8323 

2 

2 

0.0039 

0 . 9961 

2 

2 

0.0083 

0 . 9917 

2 

2 

0.0016 

0.9984 

2 

2 

0.3498 

0.6502 

2 

2 

0.0792 

0.9208 

2 

2 

0.0900 

0.9100 

2 

2 

0.0285 

0 . 9715 

2 

2 

0 .1034 

0.8966 

2 

2 

0 . 0851 

0 . 9149 

2 

2 

0.0998 

0.9002 

2 

2 

0 .4097 

0.5903 

2 

2 

0 .0946 

0 . 9054 

2 

2 

0.0285 

0 . 9715 

2 

2 

0.3352 

0 . 6648 

2 

2 

0 .4662 

0.5338 

2 

1 

0.9547 

0 . 0453 

Table  6  Classification  results  from  Table  5  summarized 
as  a  forecast  contingency  table.  In  this  case, 
the  HSS  is  0.79. 


Observed 

Observed 

Developer 

Non -developer 

Forecast 

Developer 

16 

2 

Forecast 

3 

29 

False  alarm 

Table  7 


Forecast  contingency  table  for  the  relevant 
discriminant  function  as  applied  to  independent 
data  from  the  UKMET  12-h  model.  In  this  case, 
the  HSS  is  0.55. 


Observed 

Developer 

Observed 

Non-developer 

Forecast 

Developer 

15 

7 

Forecast 

False  alarm 

4 

24 

In  both  of  the  above  scenarios  the  skill  scores 
were  calculated  using  a  classification  threshold  of  0.5. 
For  the  purpose  of  calculating  hit  and  false-alarm  rates  to 
generate  ROC  curves,  contingency  tables  were  produced  for 
the  independent  data  by  adjusting  the  classification 
threshold  from  0  to  1  by  increments  of  0.1  (Table  8)  .  At  a 
threshold  of  zero,  every  case  is  classified  to  be  a 
developer,  and  at  a  threshold  of  1.0,  every  case  is 
classified  as  a  false  alarm.  The  hit  and  false-alarm  rates 
resulting  from  the  thresholds  between  these  two  extremes 
were  plotted  to  form  a  ROC  curve  (Figure  7) .  In  this  UKMET 
12-h  case,  the  point  plotted  for  the  threshold  of  0.4  is 
the  farthest  from  the  diagonal  and  is  therefore  regarded  as 
the  optimal  threshold. 
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Table  8 


Forecast  contingency  tables  for  the  relevant 
discriminant  function  as  applied  to  independent 
data  from  the  UKMET  12-h  model.  Each  two-row 
block  gives  the  results  produced  using  the 
threshold  listed  in  column  1 .  The  last  column 
gives  the  resulting  hit  and  false-alarm  rates 
that  were  used  to  plot  a  ROC  curve  for  the 
discriminant  function. 


Threshold 

Observed 

Developer 

Observed 

Non- Developer 

O 

o 

Forecast 

19 

31 

Hit  Rate: 

Developer 

19/19 

Forecast 

0 

0 

False-alarm 

False  alarm 

Rate:  31/31 

\ — 1 

o 

Forecast 

19 

19 

Hit  Rate: 

Developer 

19/19 

Forecast 

0 

12 

False-alarm 

False  alarm 

Rate:  19/31 

0.2 

Forecast 

18 

11 

Hit  Rate : 

Developer 

18/19 

Forecast 

1 

20 

False-alarm 

False  alarm 

Rate:  11/31 

0.3 

Forecast 

16 

O 

Hit  Rate : 

Developer 

o 

16/19 

Forecast 

3 

23 

False-alarm 

False  alarm 

Rate:  8/31 

0.4 

Forecast 

16 

Q 

Hit  Rate : 

Developer 

O 

16/19 

Forecast 

3 

24 

False-alarm 

False  alarm 

Rate:  8/31 

LQ 

o 

Forecast 

15 

7 

Hit  Rate: 

Developer 

15/19 

Forecast 

A 

24 

False-alarm 

False  alarm 

4 

Rate:  7/31 

0.6 

Forecast 

15 

4 

Hit  Rate : 

Developer 

15/19 

Forecast 

A 

27 

False-alarm 

False  alarm 

4 

Rate:  4/31 

0.7 

Forecast 

ll 

3 

Hit  Rate : 

Developer 

11/19 

Forecast 

o 

28 

False-alarm 

False  alarm 

O 

Rate:  3/31 

00 

o 

Forecast 

10 

2 

Hit  Rate : 

Developer 

10/19 

1 _ 

Forecast 

9 

29 

False-alarm 
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Threshold 

Observed 

Developer 

Observed 

Non- Developer 

False  alarm 

Rate:  2/31 

0.9 

Forecast 

Q 

2 

Hit  Rate: 

Developer 

y 

9/19 

Forecast 

10 

29 

False-alarm 

False  alarm 

Rate:  2/31 

O 

\ — 1 

Forecast 

0 

0 

Hit  Rate: 

Developer 

0/19 

Forecast 

19 

31 

False-alarm 

False  alarm 

Rate:  0/31 

b.  Results  for  12-h  Forecasts 

At  12h,  the  discriminant  functions  applied  to  all 
models  (Table  3)  include  vorticity  and  warm  core 
parameters.  The  functions  applied  to  GFS  and  NOGAPS  also 
include  the  thickness  parameter.  Functions  applied  to  the 
NOGAPS  and  UKMET  models  include  the  850-500  hPa  shear  and 
both  SLP  parameters.  The  highest  skill  score  is  attained 
by  the  application  of  the  discriminant  function  to  the 
UKMET  model.  This  score  is  0.79  using  dependent  data 
(compared  to  0.59  and  0.60  for  GFS  and  NOGAPS, 

respectively)  and  0.55  using  independent  data  (compared  to 
0.29  and  0.25  for  GFS  and  NOGAPS,  respectively)  .  The  ROC 
curve  for  the  UKMET  function  (Figure  7)  also  has  the 

largest  AUC  (0.73  compared  to  0.53  and  0.55  for  GFS  and 
NOGAPS,  respectively)  with  an  optimal  classification 
threshold  of  0.4.  Therefore,  the  methodology  applied  to 
forecasts  from  the  UKMET  model,  as  described  in  the 

previous  section,  results  in  the  most  accurate 

discrimination  between  developing  vortices  and  false 
alarms . 
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ROC  Curve:  12-h  Forecasts 


Figure  7  ROC  curves  for  12-h  forecasts  based  on  an 
independent  dataset.  The  curve  for  the 

function  applied  to  GFS  is  displayed  in  red, 
UKMET  in  green,  and  NOGAPS  in  blue. 

2.  24-h  Forecasts 

For  the  24-h  forecast  interval,  the  discriminant 
functions  applied  to  each  of  the  three  sets  model  forecasts 
(Table  3)  again  include  the  warm  core  parameter,  but  only 
the  GFS  function  includes  vorticity.  All  functions  also 
include  850-500  hPa  shear  and  the  SLP  difference.  The 
functions  applied  to  the  GFS  and  NOGAPS  models  include  a 
thickness  parameter  while  the  GFS  and  UKMET  functions 
include  the  omega  and  vapor  pressure  parameters.  The 
functions  for  the  NOGAPS  and  UKMET  models  include  SLP 
minimum. 

The  discriminant  function  applied  to  the  UKMET  model 
data  again  has  the  highest  skill  in  identifying  developing 
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vortices  using  both  dependent  data  (HSS  equal  to  0.65 
compared  to  0.60  for  GFS  and  0.53  for  NOGAPS)  and 
independent  data  (HSS  equal  to  0.42  compared  to  0.33  for 
GFS  and  0.2  6  for  NOGAPS.  The  ROC  curve  (Figure  8)  for  the 
UKMET  model  again  has  the  largest  area  (AUC  equal  to  0.64 
compared  to  0.58  for  GFS  and  .060  for  NOGAPS)  with  an 
optimal  classification  threshold  of  0.4.  Overall,  the 
discriminant  analysis  applied  to  the  UKMET  forecast  is 
marginally  more  successful  in  identifying  developing 
vortices  than  that  applied  to  NOGAPS  or  GFS. 

3.  36-h  Forecasts 

All  three  36-h  forecasts  resulted  in  discriminant 
functions  that  included  the  warm  core  and  omega  parameters 
(Table  3)  .  In  addition,  functions  applied  to  the  GFS  and 
NOGAPS  models  include  vorticity,  850-200  hPa  shear,  and  SLP 
minimum.  The  functions  applied  to  the  NOGAPS  and  UKMET 
models  also  utilize  the  thickness  parameter. 
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ROC  Curve:  24-h  Forecasts 


Figure  8  ROC  curves  as  in  Figure  7,  except  for  24-h 
forecasts . 

As  for  the  12-h  and  24-h  forecasts,  the  function 
applied  to  the  UKMET  model  data  has  the  most  skill  in 
discriminating  between  developing  vortices  and  false 
alarms.  While  the  skill  scores  for  the  functions  using 
dependent  data  were  not  significantly  different  among  the 
three  models  (0.51,  0.58,  0.63  for  GFS ,  NOGAPS  and  UKMET, 

respectively) ,  the  skill  scores  using  independent  data  had 
a  much  larger  range  (0.19,  0.28,  0.43,  respectively) .  The 
AUC  for  the  UKMET  ROC  curve  (Figure  9)  is  0.60  which  is 
only  slightly  more  than  that  for  NOGAPS  (0.59)  and  GFS 
(0.57) .  The  optimal  threshold  for  the  UKMET  function  is 
0.5.  Therefore,  the  discriminant  function  applied  to  UKMET 
model  forecasts  again  exhibits  marginally  increased  skill 
in  identifying  developing  vortices. 
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ROC  Curve:  36 -h  Forecasts 


Figure  9  ROC  curves  as  in  Figure  7,  except  for  36-h 
forecasts . 

4.  48-h  Forecasts 

For  the  48-h  forecasts,  the  discriminant  functions 
applied  to  all  models  once  again  include  the  warm  core 
parameter  (Table  3)  .  The  functions  for  the  GFS  and  NOGAPS 
models  include  the  SLP  difference.  The  functions  for  the 
NOGAPS  and  UKMET  models  include  vapor  pressure  and  SLP 
minimum.  The  functions  for  the  GFS  and  UKMET  models  include 
vorticity . 

The  function  applied  to  the  UKMET  forecast  again 
attained  the  highest  skill  score.  The  HSS  was  0.80  when 
applied  to  dependent  data  and  0.56  when  applied  to 
independent  data.  By  comparison,  these  scores  were  0.48 
and  0.35  for  the  GFS  function  and  0.65  and  .034  for  the 
NOGAPS  function.  The  ROC  curve  for  the  UKMET  function 
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(Figure  10)  also  has  the  largest  area.  The  AUC  was  0.75  as 
compared  to  0.60  for  the  GFS  function  and  0.55  For  the 
NOGAPS  function.  The  optimal  classification  threshold  for 
the  UKMET  function  was  0.5.  Overall  at  48  hours,  the 
methodology  applied  to  forecasts  from  the  UKMET  model 
results  in  the  most  accurate  discrimination  between 
developing  vortices  and  false  alarms. 

ROC  Curve:  48 -h  Forecasts 


Figure  10  ROC  curves,  as  in  Figure  7,  except  for  48-h 
forecasts . 

5.  60-h  Forecasts 

All  functions  for  the  60-h  forecasts  include  vorticity 
and  vapor  pressure  (Table  3) .  The  functions  applied  to  the 
GFS  and  NOGAPS  forecasts  include  the  SLP  difference  while 
the  functions  applied  to  the  NOGAPS  and  UKMET  forecasts 
include  the  thickness  parameter  and  the  SLP  minimum. 
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The  discriminant  function  applied  to  the  UKMET  model 
has  significantly  more  skill  in  identifying  developing 
vortices  than  the  other  functions.  Using  dependent  data, 
the  HSS  was  0.93  as  compared  to  only  0.13  and  0.39  for  the 
functions  applied  to  the  GFS  and  NOGAPS  models.  Using 
independent  data,  the  UKMET  score  was  0.45  as  compared  to 
0.16  and  0.07  for  the  functions  applied  to  the  GFS  and 
NOGAPS  models.  While  the  ROC  curves  (Figure  11)  are  not  as 
widely  spread  as  the  skill  scores,  the  curve  for  the  UKMET 
function  (with  an  optimal  classification  threshold  of  0.6) 
still  has  an  AUC  of  0.73  compared  to  only  0.55  and  0.51  for 
GFS  and  NOGAPS,  respectively.  Overall,  the  discriminant 
analysis  applied  to  the  UKMET  model  is  significantly  more 
successful  in  identifying  developing  vortices. 

ROC  Curve:  60-h  Forecasts 


Figure  11  ROC  curves,  as  in  Figure  7,  except  for  60-h 
forecasts . 
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6.  72-h  Forecasts 

Using  72-h  model  forecasts,  all  discriminant  functions 
include  the  warm  core  and  850-200  hPa  shear  (Table  3) .  The 
functions  applied  to  the  GFS  and  NOGAPS  models  include  the 
SLP  minimum.  The  functions  applied  to  the  GFS  and  UKMET 
models  include  vorticity  and  850-500  hPa  shear.  The 
functions  applied  to  the  GFS  and  NOGAPS  models  include 
vapor  pressure. 

The  function  applied  to  the  NOGAPS  forecast  has  more 
skill  than  the  GFS  and  UKMET  functions.  The  HSS  was  0.67 
using  dependent  data  (compared  to  0.42  for  GFS  and  0.61  for 
UKMET)  and  0.40  using  independent  data  (compared  to  0.16 
for  GFS  and  0.31  for  UKMET)  .  While  the  skill  scores  for 

the  functions  using  both  the  dependent  and  independent  data 
spanned  a  wide  range,  the  ROC  curves  were  closely  grouped. 
The  ROC  curve  (Figure  12)  for  the  discriminant  function 
applied  to  NOGAPS  had  the  largest  AUC  (0.60)  but  this  is 
only  slightly  more  than  the  curves  for  the  GFS  (0.56)  or 
UKMET  (0.58)  functions.  The  optimal  classification 
threshold  for  the  NOGAPS  function  is  0.6.  Overall,  the 
function  for  the  NOGAPS  forecast  has  slightly  more  skill  in 
identifying  developing  vortices  at  72h. 

7.  84-h  Forecasts 

For  the  84-h  forecasts,  the  discriminant  functions  for 
all  models  include  the  warm  core  parameter  and  vapor 

pressure  parameters  (Table  3)  .  The  functions  applied  to 

the  GFS  and  NOGAPS  models  include  the  850-200  hPa  shear. 

The  functions  applied  to  the  NOGAPS  and  UKMET  models 
include  vorticity. 
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ROC  Curve:  72 -h  Forecasts 


Figure  12  ROC  curves  as  in  Figure  7,  except  for  72-h 
forecasts . 

The  discriminant  function  applied  to  the  UKMET  model 
forecasts  has  a  significantly  higher  HSS  using  both 
dependent  /  independent  data  (0.81  /  0.48,  respectively, 

compared  to  0.51  /  0.27  for  GFS  and  0.75  /  0.32  for 
NOGAPS) .  The  ROC  curves  (Figure  13)  are  closely  grouped 
for  the  three  functions,  but  the  UKMET  function  still  has 
the  largest  AUC  at  0.66  compared  to  0.64  for  the  functions 
applied  to  both  GFS  and  NOGAPS.  Overall,  the  function 
applied  to  UKMET  model  data  exhibits  the  most  skill  at  84h. 
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ROC  Curve:  84-h  Forecasts 


Figure  13  ROC  curves  as  in  Figure  7,  except  for  84-h 
forecasts . 

8.  96-h  Forecasts 

By  96h,  the  number  of  developing  cases  identified  in 
each  model  has  decreased  appreciably.  Only  one  developer 
was  identified  in  the  UKMET  96-h  forecast  which  results  in 
an  unrealistic  discriminant  function  for  UKMET  that 
includes  only  one  parameter.  Although  the  function  has  a 
perfect  skill  score  using  the  dependent  data,  when  that  one 
case  is  removed,  the  function  has  a  skill  score  and  AUC  of 
zero  using  'independent'  data.  The  corresponding  ROC 

curve  is  therefore  not  included  in  Figure  14.  The  NOGAPS 
and  GFS  forecasts  still  have  enough  developers  to  make  the 
discriminant  analysis  well-posed,  however,  as  the  forecast 
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interval  increases,  the  resulting  statistics  are  presumed 
to  be  less  meaningful  given  the  decrease  in  the  size  of  the 
data  set. 

The  only  parameters  included  in  the  discriminant 
functions  for  both  the  GFS  and  NOGAPS  models  are  omega  and 
the  SLP  difference.  The  discriminant  function  applied  to 
the  GFS  model  (with  scores  of  0.50  and  0.38)  exhibits  more 
skill  in  identifying  developing  vortices  than  that  of 
NOGAPS  (with  scores  of  .32  and  0.21)  .  The  ROC  curve 
(Figure  14)  for  the  GFS  function  also  has  a  slightly  larger 
AUC  (0.62)  than  that  of  NOGAPS  (0.55).  In  summary,  the 
methodology  applied  to  the  GFS  forecast  is  the  most 
successful  in  identifying  developing  vortices  at  96h. 

ROC  Curve:  96 -h  Forecasts 


Figure  14  ROC  curves  as  in  Figure  7,  except  for  96-h 
forecasts  and  the  curve  for  the  UKMET  function 
is  omitted. 
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9.  108-h  Forecasts 

The  NOGAPS  108-h  forecast  had  only  one  developing 
system,  therefore  the  methodology  does  not  provide 
realistic  results.  This  ROC  curve  is  not  included  in 
Figure  15.  The  discriminant  function  applied  to  the  UKMET 
model,  which  is  based  on  significantly  fewer  cases  than  the 
GFS  function,  attains  a  skill  score  of  1  using  independent 
data  while  the  latter  attains  a  skill  score  of  0.60.  The 
score  for  the  UKMET  function,  however,  decreases  to  0.31 
using  independent  data,  which  is  equal  to  that  for  the  GFS 
function.  The  UKMET  function  has  the  largest  AUC  (Figure 
15)  at  0.67,  which  is  only  slightly  larger  than  the  AUC  for 
the  function  applied  to  the  GFS  model  (.58)  .  Overall,  the 
functions  applied  to  UKMET  and  GFS  model  data  display 
roughly  equivalent  skill  at  108h. 


ROC  Curve:  108-h  Forecasts 


False  Alarm  Rate 

Figure  15  ROC  curves  as  in  Figure  7,  except  for  108-h 
forecasts  and  the  curve  for  the  NOGAPS 
function  is  omitted. 


45 


10.  120-h  Forecasts 

Very  few  120-h  forecasts  of  developers  exist  in  any  of 
the  models.  Since  there  are  only  two  developers  in  the 
NOGAPS  forecast,  the  methodology  applied  to  this  model  is 
not  meaningful.  The  ROC  curve  for  the  NOGAPS  function  is 
therefore  not  included  in  Figure  16.  The  discriminant 
functions  applied  to  the  GFS  and  UKMET  forecasts  include 
the  warm  core,  850-500  hPa  shear,  and  omega  parameters 
(Table  3)  .  The  function  applied  to  the  UKMET  model  has 
higher  skill  scores  (0.72  and  0.42)  than  the  function 
applied  to  the  GFS  model  (0.55  and  0.10)  and  the  ROC  curve 
(Figure  16)  for  the  UKMET  function  has  a  larger  AUC  (0.65) 
than  that  of  the  GFS  function  (0.55) . 


ROC  Curve:  120-h  Forecasts 


Figure  16  ROC  curves  as  in  Figure  7,  except  for  120-h 
forecasts  and  the  curve  for  the  NOGAPS 
function  is  omitted. 
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C .  CLASSIFICATION  EXAMPLES 

In  this  section,  two  cases  are  examined  to  provide  an 
example  of  how  the  methodology  would  be  interpreted  with 
respect  to  a  single  tropical  vortex  that  is  forecast  to 
exceed  vorticity  and  warm  core  thresholds. 

1 .  Correctly  Classified  False  Alarm 

This  section  examines  the  model  data,  statistics,  and 
classification  of  a  case  from  the  UKMET  48-h  model.  The 
vortex,  which  was  tracked  in  the  TCVTP  from  17°N,  15°W  at 

1200UTC  7  July  2005,  was  assigned  the  ID 

"UKM_2  00507  07 1 2_1 7_0 15"  by  the  database.  The  48-h  UKMET 
forecast  to  be  discussed  was  initiated  at  1200UTC  14  July 
2005.  The  48-h  vorticity  and  warm  core  values  for  this 
vortex  are  identified  on  Figure  17  with  a  blue  arrow.  The 
sequence  of  forecast  tracks  for  this  vortex  is  shown  in 
Figure  18.  The  48-h  track  forecast  is  illustrated  in 
Figure  1 9 . 

The  UKMET  forecast  and  analyzed  values  for  the  six 
parameters  used  in  the  48-h  discriminant  function  (Tables  3 
and  4)  are  compared  in  Figures  20-25.  Figure  20 


illustrates 

that 

the 

48-h 

forecast 

relative 

vorticity  was 

higher  than 

the 

analyzed 

relative 

vorticity 

Figure  21 

illustrates 

that 

the 

48-h 

forecast 

warm  core  was  higher 

than  the  analyzed  warm  core.  Figure  22  illustrates  that 
the  48-h  forecast  wind  shear  was  lower  than  the  analyzed 
wind  shear.  Figure  23  illustrates  that  the  48-h  forecast 
vertical  motion  was  higher  than  the  analyzed  vertical 
motion.  Figure  24  illustrates  that  the  48-h  forecast  vapor 
pressure  was  higher  than  the  analyzed  vapor  pressure. 
Figure  25  illustrates  that  the  48-h  forecast  sea-level 
pressure  was  lower  than  the  analyzed  sea-level  pressure. 
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700-500  hPa  Warm  Core  (°C) 


UKM  48-h  Forecast  Avg.  850-500  hPa  Vorticity  and  700-500  hPa  Warm  Core 


-2  o  2  4  6  8  10 


850-500  hPa  Relative  Vorticity  (10'5  s*1) 

Figure  17  Scatterplot  of  850-500  hPa  relative  vorticity 
and  700-500  hPa  warm  core  for  all  vortices  in 
the  TCVTP  database  for  the  UKMET  model  at  the 
48-h  forecast  interval.  The  case 

UKM_2005070712_17_015  is  highlighted  with  a 
blue  arrow.  The  case  UKM_2005091900_018_002 
is  highlighted  with  a  green  arrow. 
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Figure  18  The  analyzed  track  (black)  and  all  UKMET 
forecast  tracks  (colors)  for  the  case 
UKM  2005070712  17  015. 
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Figure  19  As  in  Figure  18,  except  with  the  UKMET 
forecast  track  initiated  at  1200  UTC  14  July 
2005  (blue)  and  the  48-h  analyzed  and  forecast 
positions  circled  in  red. 
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UKM  200507071 2_17_01 5:  850-500  hPa  Relative  Vorticity 


—•—Forecast:  0000  UTC  15  July  2005  —■— Analysis 


Figure  20  Average  relative  vorticity  (10~5  s_1)  between 

850  and  500  hPa  for  case 

UKM_2  00507071 2_1 7_0 1 5 .  The  UKMET  48-h 

forecast  values  from  1200  UTC  14  July  2005  are 
plotted  with  red  circles,  while  analyzed 
values  are  plotted  with  blue  squares. 


UKM  2005070712  17  015:  700-500  hPa  Warm  Core 


Forecast:  0000  UTC  15  July  2005  — Analysis 


Figure  21  As  in  Figure  20,  except  for  warm  core  (°C)  at 
700-500  hPa . 
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UKM  2005070712  17  015:  200-850  hPa  Wind  Shear 


—•—Forecast:  0000  UTC  15  July  2005  Analysis 

Figure  22  As  in  Figure  20,  except  for  vertical  wind 
shear  (m  s-1)  between  200  and  850  hPa. 


UKM  2005070712  17  015:  500  hPa  Vertical  Motion 


-•—Forecast:  0000  UTC  15  July  2005  -■— Analysis 


Figure  23 


As  in  Figure  20,  except  for  vertical  motion 
(m  s”1  *  10)  at  500  hPa. 
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UKM  200507071 2_17_01 5:  700-500  hPa  Vapor  Pressure 


—♦—Forecast:  0000  UTC  15  July  2005  — Analysis 

Figure  24  As  in  Figure  20,  except  for  vapor  pressure 
(mb)  between  700  and  500  hPa. 


UKM  2005070712  17  015:  Sea-Level  Pressure 


-♦—Forecast:  0000  UTC  15  July  2005  -■- Analysis 


Figure  25  As  in  Figure  20,  except  for  sea-level  pressure 
(mb)  . 

Table  9  is  a  list  of  the  values  of  the  six  parameters 
for  this  case  (Tables  3  and  4)  compared  to  the  overall  mean 

values  for  the  developer  and  false-alarm  classifications. 
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The  values  of  four  of  the  six  parameters  for  this  case  were 
closer  to  the  mean  values  for  false  alarms.  As  defined  by 
Equation  (6)  in  Chapter  II,  Section  D,  the  distance  between 
each  value  and  the  midpoint  of  the  two  classification 
groups  for  each  parameter  contributes  to  the  computation  of 
the  probabilities  of  group  membership.  As  calculated  by 
the  discriminant  analysis,  the  probability  that  this  case 
belongs  to  Group  1  is  0.0003  and  the  probability  that  it 

belongs  to  Group  2  is  .9997.  The  discriminant  function 

therefore  correctly  classified  this  case  as  a  false  alarm. 

Table  9  Six  parameters  for  case  UKM_2005070712_17_015 
(third  row)  compared  to  the  mean  values  for 
developers  (first  row)  and  false  alarms  (second 
row)  .  The  fourth  and  fifth  rows  contain  the 

differences  between  the  values  for  this  case 

and  the  mean  values  for  developers  and  false 
alarms,  respectively.  The  smaller  differences 
are  highlighted. 


Parameter 

(as  in  Table  4) 

1 

2 

4 

6 

7 

9 

Mean  for 
developers 

3.76 

0.13 

0.85 

2.05 

64.73 

1010 . 4 

Mean  for  false 
alarms 

4.3 

0.41 

0.34 

1 . 8 

60.4 

1012.2 

UKM  2005070712 

17  015 

4 .464 

0.96 

3.54 

3.94 

57.82 

1013.3 

Difference  from 
developers 

0.70 

0.83 

2.7 

1 . 9 

-6.9 

2 . 9 

Difference  from 
false  alarms 

0 . 17 

0.58 

3.3 

2.2 

-2 . 66 

1.12 

2 .  Incorrectly  Classified  False  Alarm 

This  section  examines  a  tropical  vortex  that  was 
initially  identified  in  the  UKMET  48-h  forecast  at  18°N, 
2  °W  at  0000UTC  19  September  2005  ("UKM_20050919_018_002")  . 

The  48-h  UKMET  forecast  to  be  examined  was  initiated  at 
1200  UTC  23  September  2005.  Vorticity  and  warm  core  values 
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for  this  vortex  are  identified  on  Figure  17  with  a  green 
arrow.  The  sequence  of  forecast  tracks  for  this  vortex  is 
shown  in  Figure  26.  The  48-h  forecast  track  is  illustrated 
in  Figure  27 . 
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Figure  26  The  analyzed  track  (black)  and  all  UKMET 
forecast  tracks  (colors)  for  the  case 
UKM  20050919  018  002. 


Figure  27  The  analyzed  track  (black)  and  UKMET  forecast 
track  initiated  at  1200  UTC  23  September  2005 
(blue)  for  case  UKM_20050919_018_002 .  The  48- 
h  analyzed  and  forecast  positions  are  circled 
in  red. 
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The  UKMET  forecast  and  analyzed  values  for  the  six 
parameters  used  in  the  48-h  discriminant  function  (Tables  3 
and  4)  are  compared  in  Figures  28-33.  Figure  28 
illustrates  that  the  48-h  forecast  relative  vorticity  was 
higher  than  the  analyzed  relative  vorticity.  Figure  29 
illustrates  that  the  48-h  forecast  warm  core  was  lower  than 
the  analyzed  warm  core.  Figure  30  illustrates  that  the 
48-h  forecast  wind  shear  was  lower  than  the  analyzed  wind 
shear.  Figure  31  illustrates  that  the  48-h  forecast 
vertical  motion  was  higher  than  the  analyzed  vertical 
motion.  Figure  32  illustrates  that  the  48-h  forecast  vapor 
pressure  was  higher  than  the  analyzed  vapor  pressure. 
Figure  33  illustrates  that  the  48-h  forecast  sea-level 
pressure  was  equal  to  the  analyzed  sea-level  pressure. 


UKM  2005091 900_018_002:  850-500  hPa  Relative  Vorticity 


Forecast:  0000  UTC  23  Sep  05  Analysis 


Figure  28  As  in  Figure  20,  except  for  case 

UKM_2  0050  91 9_0 1 8  _002.  The  UKMET  48-h 

forecast  values  from  1200  UTC  23  September 
2005  are  plotted  with  red  circles  while 

analyzed  values  are  plotted  with  blue  squares . 
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UKM  2005091900  018  002:  700-500  hPa  Warm  Core 


-•—Forecast:  0000  UTC  23  Sep  05  Analysis 


Figure  29  As  in  Figure  28,  except  for  warm  core  (°C)  at 
700-500  hPa . 


UKM  2005091900  018  002:  200-850  hPa  Vertical  Wind  Shear 


•  Forecast  0000  UTC  23  Sep  05  ■  Analysis 


Figure  30  As  in  Figure  28,  except  for  vertical  wind 
shear  (m  s_1)  between  200  and  850  hPa. 
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UKM  2005091900  018  002:  500  hPa  Vertical  Motion 


-•—Forecast:  0000  UTC  23  Sep  05  -■— Analysis 


Figure  31  As  in  Figure  28,  except  for  vertical  motion 
(m  s_1  *  10)  at  500  hPa. 


UKM  2005091 91 2_018_002:  700-500  hPa  Vapor  Pressure 


— •— Seriesl  -■— Analysis 


Figure  32  As  in  Figure  28,  except  for  vapor  pressure (mb) 
between  700  and  500  hPa. 
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UKM  2005091912  018  002:  Sea-level  Pressure 


-•—Forecast:  0000  UTC  23  Sep  05  -■— Analysis 


Figure  33  As  in  Figure  28,  except  for  sea-level  pressure 
(mb)  . 

Table  10  is  a  list  of  the  values  of  the  six  parameters 
for  this  case  compared  to  the  overall  mean  values  for  the 
developer  and  false-alarm  classifications.  Values  of  five 
of  the  six  parameters  for  this  case  were  closer  to  the  mean 
values  for  developers  than  to  the  mean  values  for  false 
alarms.  As  calculated  by  the  discriminant  analysis,  the 
probability  that  this  case  belongs  to  Group  1  is  0.84  and 
the  probability  that  it  belongs  to  Group  2  is  0.16.  The 
discriminant  function  therefore  incorrectly  classified  this 
case  as  a  developer.  The  cause  of  this  misclassif ication 
can  be  seen  in  Figures  28,  30,  and  31  which  demonstrate 

that  through  the  48-h  forecast  there  was  a  strong 
indication  that  the  vortex  would  develop  into  a  tropical 
cyclone.  While  the  model  would  eventually  weaken  the 

vortex  beyond  48h,  it  did  so  12h  after  the  actual  vortex 
weakened . 
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Table  10 


Six  parameters  for  case  UKM_20050919_018_002 
(third  row)  compared  to  the  mean  values  for 
developers  (first  row)  and  false  alarms  (second 
row)  .  The  fourth  and  fifth  rows  contain  the 
differences  between  the  values  for  this  case 
and  the  mean  values  for  developers  and  false 
alarms,  respectively.  The  smaller  differences 
is  highlighted. 


Parameter 

(as  in  Table  4) 

i 

2 

4 

6 

7 

9 

Mean  for 
developers 

3.76 

0.13 

0.85 

2.05 

64.73 

1010 . 4 

Mean  for  false 

alarms 

4.3 

0 .41 

0.34 

1.8 

60.4 

1012.2 

UKM  20050919 

018  002 

4.33 

0.25 

1.19 

2 . 6 

6  6.4 

1010.3 

Difference  from 
developers 

0.57 

0.12 

0.34 

0.56 

1.63 

1 

o 

o 

Difference  from 

false  alarms 

0.032 

-0 .16 

0.88 

0.82 

6.2 

-1 . 99 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


As  the  NHC  and  JTWC  issue  tropical  cyclone  advisories 
for  longer  forecast  periods,  the  necessity  to  predict 
tropical  cyclone  formation  becomes  more  important.  Many  of 
the  processes  that  are  believed  to  control  the  timing  of 
tropical  cyclone  formation  occur  in  the  mesoscale. 
Therefore,  it  is  a  matter  of  current  debate  as  to  whether 
global  model  forecasts  contain  valuable  information  with 
regard  to  tropical  cyclone  formation.  However,  the  current 
circumstances  are  such  that  the  global  models  are  the  only 
numerical  data  routinely  available  to  forecasters  for  this 
type  of  prediction.  Therefore,  the  goal  of  this  study  was 
to  use  the  global  model  data  available  in  the  TCVTP 
database  to  develop  a  tool  to  aid  in  the  prediction  of 
tropical  cyclone  formation. 

After  a  preliminary  examination  of  forecasts  from  the 
three  models,  it  became  clear  that  the  models  could  not 
effectively  distinguish  developing  vortices  from  the  large 
number  of  non-developing  vortices  (Appendix  B)  .  As  a 
result,  the  focus  was  shifted  to  only  those  cases  in  which 
a  model  forecasted  a  vortex  to  exceed  vorticity  and  warm- 
core  thresholds  for  tropical  cyclone  formation.  Hence,  the 
objective  was  to  develop  a  procedure  for  determining  which 
model-predicted  tropical  cyclones  will  actually  develop  and 
which  are  false  alarms. 

In  this  study,  nine  parameters  relevant  to  tropical 
cyclone  formation  were  subjected  to  a  linear  discriminant 
analysis.  The  nine  parameters  derived  from  global  model 
output  do  not  define  all  tropical  cyclone  characteristics. 
Vortices  are  identified  in  which  vorticity  and  warm  core 
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thresholds  are  exceeded,  but  in  which  other  parameters 
(e.g.,  wind  speed)  suggest  that  the  vortex  is  not  a 
tropical  cyclone.  Correct  forecasts  of  vorticity  and  warm 
core  parameters  for  these  types  of  vortices  are  not 

considered  false  alarms  since  the  forecasts  did  verify  in 
the  correct  category. 

The  discriminant  analysis  was  performed  on  the  nine 

parameters  from  each  model  at  each  forecast  period  (30 
analyses  in  total)  to  determine  the  combination  of 
parameters  that  best  discriminates  between  a  vortex  that 

was  correctly  forecast  to  intensify  into  a  tropical  cyclone 
(developer)  and  a  vortex  that  was  forecast  to  intensify 
into  a  tropical  cyclone,  but  did  not  (false  alarm)  .  The 
performance  of  the  resulting  discriminant  functions  were 
then  assessed  using  the  Heidke  Skill  Score  and  Receiver 
Operating  Characteristic  curves. 

A.  CONCLUSIONS 

The  primary  conclusion  of  this  thesis  is  that  linear 
discriminant  analysis  applied  to  the  parameters  defined  in 
the  TCVTP  database  provides  a  valid  method  of 
discriminating  between  developers  and  false  alarms  in  the 
global  model  data.  The  resulting  discriminant  functions 

exhibited  various  levels  of  skill.  In  27  of  30  cases,  the 
discriminant  function  performed  better  than  a  random 
forecast  as  measured  by  the  HSS  and  the  area  under  the  ROC 
curve.  It  is  important  to  note  that  the  VORTRACK  system 
and  the  TCVTP  database  make  this  type  of  data  analysis 
quick  and  simple  to  complete. 

Efforts  to  reduce  the  number  of  parameters  considered 
in  the  analysis  (by  considering  PDFs,  for  example),  did  not 
yield  profitable  results.  The  discriminant  functions 
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resulting  from  each  of  the  thirty  analyses  used  different 
combinations  of  the  nine  parameters  (Table  3)  .  It  is 
therefore  concluded  that  all  possible  combinations  of  all 
available  parameters  should  be  considered  in  a  discriminant 
analysis . 

Some  challenges  remain  to  performing  this  type  of  data 
analysis.  Models  undergo  frequent  updates.  Each  time 
changes  are  made  to  the  model,  a  new,  and  sufficient , 
training  sample  would  be  required  before  the  discriminant 
analysis  could  yield  meaningful  results.  Additionally,  the 
small  number  of  developing  vortices  identified  and  tracked 
by  the  TCVTP  at  the  longer  forecast  intervals  poses  a 
problem.  For  example,  not  enough  developing  vortices  were 
tracked  in  the  2005  UKMET  forecasts  for  the  discriminant 
analysis  to  produce  meaningful  results  beyond  84h. 

The  discriminant  function  applied  to  the  UKMET  model 
outperformed  those  applied  to  NOGAPS  and  GFS  in  eight  of 
the  10  forecast  periods.  This  superiority  is  at  least 
partially  attributable  to  a  low  false-alarm  rate  associated 
with  the  UKMET  model.  However,  the  UKMET  model  also  has  a 
tendency  for  a  low  probability  of  detection  which  is 
responsible  for  the  lack  of  cases  at  the  longer  forecast 
intervals.  The  GFS  is  the  only  model  in  which  the  TCVTP 
identifies  sufficient  vortices  at  all  forecast  intervals. 
However,  the  HSS  and  ROC  curves  for  GFS  suffer  as  a  result 
of  its  tendency  to  over-predict  formation  (i.e.,  higher 
false-alarm  rate)  . 

B.  RECOMMENDATIONS  FOR  FUTURE  WORK 

This  analysis  should  be  replicated  for  different 
datasets  to  include  additional  geographic  regions  and  time 
periods.  If  future  analysis  demonstrates  that  the  models 
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are  becoming  capable  of  discriminating  between  developers 
and  false  alarms,  the  methodology  should  be  redirected 
toward  that  important  forecast  problem. 

Two  extensions  to  the  discriminant  analysis  may  lead 
to  improved  results.  First,  all  parameters  in  this  study 
were  considered  in  a  static  sense.  However,  the  time 
variability,  or  trend,  of  parameters  could  be  incorporated 
and  may  have  great  potential  to  increase  the  skill  of  the 
discriminant  analysis.  Second,  the  analysis  could  consider 
combinations  of  parameters  from  multiple  models  that  would 
result  in  an  'ensemble'  discriminant  function.  For 

example,  the  discriminant  function  for  the  12-h  forecast 
period  may  include  the  vorticity  parameter  from  the  GFS 
model,  the  warm-core  parameter  from  the  NOGAPS  model,  and 
the  vapor  pressure  parameter  from  the  UKMET  model. 
However,  the  probability  of  detection  for  the  UKMET  and  GFS 
models  would  need  to  increase  for  an  ensemble  approach  to 
be  useful  at  longer  forecast  intervals. 
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700-500  hPa  Warm  Core  (°C) 


APPENDIX  A.  SCATTERPLOTS  FOR  GFS  12 -H  FORECAST 


GFS  1 2-h  Forecast  Avg.  S50-500  hPa  Vorticity  and  700-500  hPa  Warm  Core 
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700-500  hPa  Warm  Core  (°C)  700-500  hPa  Warm  Core  (°C) 


GFS  1 2-h  Forecast  Avg.  850-500  hPa  Vorticity  and  700-500  hPa  Warm  Core 
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850-500  hPa  Relative  Vorticity  (ID-5  s"1 ) 
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700-500  hPa  Warm  Core  (°C)  700-500  hPa  Warm  Core  (°C) 
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GFS  1 2-h  Forecast  Avg.  850-500  hPa  Vorticity  and  700-500  hPa  Warm  Core 

1.4 
1.2 

i 

0.8 
0.6 

0.4 

0.2 
0 

-0.2 
-0.4 

-0.6 

-2  0  2  4  6  8  10 

850-500  hPa  Relative  Vorticity  (1 0"5  s"1 ) 

67 


700-500  hPa  Warm  Core  (°C)  700-500  hPa  Warm  Core  (°C) 
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700-500  hPa  Warm  Core  (°C) 


APPENDIX  B.  SCATTERPLOTS  INCLUDING  OMEGA 


GFS  1 2-h  Forecast  Avg.  850-500  hPa  Vorticity  and  700-500  hPa  Warm  Core 
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APPENDIX  C.  PROBABILITY  DISTRIBUTION  FUNCTIONS 
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GFS:  24-h  Parameter  Distributions 
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GFS:  36-h  Parameter  Distributions 
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GFS:  48-h  Parameter  Distributions 
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GFS:  72-h  Parameter  Distributions 
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GFS:  84-h  Parameter  Distributions 
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GFS:  96-h  Parameter  Distributions 
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GFS:  120-h  Parameter  Distributions 


0  15  30  45  60  75  90  105 

700-500  hPa  Vapor  pressure  (mb) 

Non  Developers 


-6  -5  -4  -3  -2-10  1 

Sea-Level  Press.  Diff  (mb) 

Developers 


0.2 
0.15 
0.1 
0.05 
0 

990  9951 000  0051 01 0  01 5  020  025 

Sea-Level  Pressure  (mb) 

False  Alarms 


m 


NGP:  12-h  Parameter  Distributions 
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NGP:  24-h  Parameter  Distributions 
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NGP:  36-h  Parameter  Distributions 
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NGP:  48-h  Parameter  Distributions 
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NGP:  60-h  Parameter  Distributions 


0.8 
0.6 
0.4 
0.2 
0 

-4  -2  0  2  4  6  3  10 

850-500  hPa  RVor  (10'5  s'1) 


-1  -0.5  0  0.5  1  1.5  2  2.5 

700-500  hPa  Warm  Core  (°Q 


-10  -5  0  5  10  15  20  25 

850-500  hPa  Shear  (m  s'1) 


0  15  30  45  60  75  90  105 

700-500  hPa  Vapor  pressure  (mb) 

Non  Developers 


-6  -5  4  -3  -2-10  1 

Sea-Level  Press.  Diff  (mb) 


Developers 


ol — :  ^  ' — ■ — — 

990  9951 000  0051 01 0  01 5  020  025 

Sea-Level  Pressure  (mb) 


False  Alarms 


92 


NGP:  72-h  Parameter  Distributions 
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NGP:  84-h  Parameter  Distributions 
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NGP:  96-h  Parameter  Distributions 
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NGP:  108-h  Parameter  Distributions 
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NGP:  120-h  Parameter  Distributions 
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UKM:  12-h  Parameter  Distributions 
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UKM:  24-h  Parameter  Distributions 
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UKM:  36-h  Parameter  Distributions 
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UKM:  48-h  Parameter  Distributions 
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UKM:  72-h  Parameter  Distributions 
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UKM:  96-h  Parameter  Distributions 
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UKM:  108h  Parameter  Distributions 
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UKM:  120-h  Parameter  Distributions 
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